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(54) Image processing method and apparatus 

(57) Face information is detected from each of orig- 
inal imagejsignals, each of the original image signals 
represent^ a person image, in which a face pattern of 
a person ^embedded. The face information represents 
a position and/or a size of the face pattern of the person 
in the person image represented by each original image 
signal. A face pattern normalizing process is performed 
on each of the original image signals and in accordance 
with the detected face information, and a plurality of nor- 
malized image signals are obtained from the face pat- 
tern normalizing process. A plurality of images, which 
are represented by the normalized image signals, are 
laid out in a predetermined layout, and a layout image 
signal representing the thus formed layout image is 
obtained. A layout image, such„as a school photo, is 
thus capable of being formed efficiently. 
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Description 

BACKGROUND OF THE INVENTION 
s Field of the Invention 

[0001] This invention relates to an image processing method and apparatus for laying out person images, which 
have been obtained by photographing a plurality of persons, and forming a layout image, such as a school photo. This 
invention also relates to a recording medium, on which a program for causing a computer to execute the image process- 
io ing method has been recorded and from which the computer is capable of reading the program. 

Description of the Prior Art 

[0002] Album prints, typically school photos, are formed by laying out person images, which have been obtained by 
15 photographing persons, in predetermined layouts, thus obtaining layout images, and printing the layout images. As illus- 
trated in Figure 5, if the person images are merely laid out in a layout image comprising the person images, positions 
and/or sizes of face patterns of the persons will vary for different person images, and the layout image will become 
unsightly. Therefore, it may be considered to take photographs such that the positions and/or the sizes of the face pat- 
terns of the persons in the photographs may be uniform. However, in cases where photographs of a plurality of persons 
20 are taken, considerable time and labor are required to take the photographs by changing the position of the camera in 
accordance with the heights of the persons and/or the sizes of the faces of the persons. Also, the position of the camera 
with respect to the persons varies for different cameramen and different dates of photographing. Accordingly, it is diffi- 
cult to take the photographs such that the positions and/or the sizes of the face patterns of the persons in all of the pho- 
tographs may be uniform. 

25 [0003] Therefore, in cases where a layout image is to be formed, when operations for printing from sheets of neg- 
ative film, on which person images have been recorded, are performed, the positions of the sheets of negative film 
and/or the positions of photosensitive materials have heretofore been adjusted such that the positions and/or the sizes 
of the face patterns of the persons may be uniform among the images. Alternatively, a layout image has heretofore been 
formed by laying out person images, which have been obtained by performing a trimming process such that the posi- 

30 tions and/or the sizes of the face patterns of the persons may be uniform. As another alternative, a layout image has 
heretofore been formed by printing person images to a comparatively large size, arraying the prints of the person 
images such that the positions of the face patterns of the persons may be approximately uniform, and again performing 
the photographing operation. 

[0004] However, with the technique for forming a layout image by adjusting the positions of the sheets of negative 
35 film and/or the positions of the photosensitive materials and performing the printing operations, and with the technique 
for forming a layout image by arraying the printed images again performing the photographing operation, the efficiency 
of the layout image formation and the productivity of the layout images cannot be kept high. 

SUMMARY OF THE INVENTION 

40 

[0005] The primary object of the present invention is to provide an image processing method, wherein a layout 
image, such as a school photo, is capable of being formed efficiently. 

[0006] Another object of the present invention is to provide an apparatus for carrying out the image processing 
method. 

45 [0007] The specific object of the present invention is to provide a recording medium, on which a program for caus- 
ing a computer to execute the image processing method has been recorded and from which the computer is capable of 
reading the program. 

[0008] The present invention provides an image processing method for obtaining a layout image signal represent- 
ing a layout image, in which a plurality of person images are laid out, from a plurality of original image signals, each of 
50 the original image signals representing a person image, in which a face pattern of a person is embedded, the method 
comprising the steps of: 

i) detecting face information from each of the original image signals, the face information representing a position 
and/or a size of the face pattern of the person in the person image represented by each original image signal, 
55 ii) performing a face pattern normalizing process on each of the original image signals and in accordance with the 
detected face information, a plurality of normalized image signals being obtained from the face pattern normalizing 
process, and 

Hi) laying out a plurality of images, which are represented by the normalized image signals, in a predetermined lay- 
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out, whereby the layout image signal representing the thus formed layout image is obtained. 

n • 

[0009]= The term "face information" as used herein means, for example, the information representing a displace- 
ment quantity for transformation of a person image such that the position of the face pattern in the person image may 
5 coincide with a position in a predetermined reference image or a predetermined coordinate position. Also, the term 
"face pattern normalizing process" as used herein means a process for transforming the original image signal with an 
affine transformation technique, or the like, such that the positions and/or the sizes of the face patterns of the persons 
may become uniform among the person images. 

[0010] The present invention also provides an image processing apparatus for obtaining a layout image signal rep- 
10 resenting a layout image, in which a plurality of person images are laid out, from a plurality of original image signals, 
each of the original image signals representing a person image, in which a face pattern of a person is embedded, the 
apparatus comprising: < 

i) detection means for detecting face information from each of the original image signals, the face information rep- 
is resenting a position and/or a size of the face pattern of the person in the person image represented by each original 

image signal, 

ii) normalization means for performing a face pattern normalizing process on each of the original image signals and 
in accordance with the detected face information, a plurality of normalized image signals being obtained from the 
face pattern normalizing process, and 

20 iii) editing means for laying out a plurality of images, which are represented by the normalized image signals, in a 
predetermined layout, and obtaining the layout image signal representing the thus formed layout image. 

[001 1] The present invention further provides a recording medium, on which a program for causing a computer to 
execute the image processing method in accordance with the present invention has been recorded and from which the 

25 computer is capable of reading the program. 

[0012] With the image processing method and apparatus in accordance with the present invention, the face pat- 
terns of the persons in the person images are normalized, and the normalized image signals are thereby obtained. The 
images, which are represented by the normalized image signals, are laid out in the predetermined layout, and the layout 
image signal representing the thus formed layout image is obtained. Therefore, in the layout image reproduced from the 

30 layout image signal, the positions and/or the sizes of the face patterns of the persons become uniform. Accordingly, the 
layout image, in which the positions and/or the sizes of the face patterns of the persons are uniform, can be formed effi- 
ciently. As a result, the productivity of layout images, such as school photos, can be enhanced. 

BRIEF DESCRIPTION OF THE DRAWINGS 

35 

[0013] 

Figure 1 is a block diagram showing an album printing system, in which an embodiment of the image processing 
apparatus in accordance with the present invention is employed, 
40 Figure 2 is an explanatory view showing a person image, 

Figure 3 is a flow chart showing how the embodiment of the image processing apparatus in accordance with the 
present invention operates, 

Figure 4 is a schematic view showing a layout image, in which a plurality of person images have been normalized, 
Figure 5 is a schematic view showing a layout image, in which a plurality of person images have not been normal- 
45 ized, and 

Figure 6 is a schematic view showing layout images obtained with a different embodiment of the image processing 
apparatus in accordance with the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0014] The present invention will hereinbelow be described in further detail with reference to the accompanying 
drawings. 

[0015] Figure 1 is a block diagram showing an album printing system, in which an embodiment of the image 
processing apparatus in accordance with the present invention is employed. With reference to Figure 1 , the album print- 
55 ing system comprises reading means 1 for reading out a plurality of original image signals SO, SO, each of which 
represents a person image. The album printing system also comprises detection means 2 for detecting face information 
H from each of the original image signals SO, SO, ... having been received from the reading means 1 . The face informa- 
tion H represents a displacement quantity, which is to be utilized in a face pattern normalizing process for uniformizing 
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the positions and/or the sizes of face patterns in the person images. The album printing system further comprises nor- 
malization means 3 for performing affine transformation on each of the original image signals SO, SO, ... and in accord- 
ance with the corresponding face information H, and thereby obtaining a plurality of normalized image signals S1 , S1 , 
... representing images, in which the positions and/or the sizes of the face patterns of the persons have been normal- 
5 ized. The album printing system still further comprises editing means 4 for laying out the images, which are represented 
by the normalized image signals S1, S1 in a predetermined layout, and obtaining a layout image signal S2 repre- 
senting a layout image R, in which the plurality of the person images have been laid out. The album printing system also 
comprises a printer 5 for reproducing the layout image R from the layout image signal S2. 

[0016] The original image signals SO, SO, ... may be obtained from a read-out apparatus, such as a scanner, which 
10 reads out the person images from negative film or positive film. Alternatively, the original image signals SO, SO, ... may 
be obtained from image sensing operations performed by a digital camera. The thus obtained original image signals 
SO, SO, are stored on a storage medium or a storage device. The reading means 1 reads the original image signals SO, 
SO, ... from the storage medium or the storage device. 

[0017] The detection means 2 comprises edge detecting means 6 for detecting edges in the face pattern embed- 
15 ded in the person image, which is represented by each original image signal SO. The detection means 2 also comprises 
binarization means 7 for binarizing the original image signal SO, which has been obtained from the edge detection, and 
obtaining a binary signal SO*. The detection means 2 further comprises pattern matching means 8. The pattern match- 
ing means 8 performs a pattern matching process on the binary signal SO' and a reference signal G, which represents 
a predetermined binary reference image. The pattern matching means 8 thus detects at least two points, at which the 
20 degree of correlation between the person image and the reference image is highest. Also, the pattern matching means 
8 determines displacement quantities, which are to be utilized for transforming a center position having coordinates (x, 
y) in the face pattern embedded in the person image as illustrated in Figure 2 into a center position having coordinates 
(xO, yO) in the reference image, and uniformizing a size v of the face pattern in the vertical direction and a size h of the 
face pattern in the horizontal direction. The detection means 2 feeds information, which represents the thus determined 
25 displacement quantities, as the face information H into the normalization means 3. The reference image represents a 
contour image of a face having a predetermined size, which contour image is located such that it may have the center 
position at the predetermined coordinates (x0, yO). 

[0018] In accordance with the face information H received from the detection means 2, the normalization means 3 
performs affine transformation on the original image signal SO and obtains a normalized image signal S1. The affine 
30 transformation is performed with Formula (1) shown below. 



35 



(x2\ I a 

-b 





(i) 



in which (x1, y1) represents the coordinate values of the person image before being transformed, and (x2, y2) repre- 
sents the coordinate values of the person image after being transformed. 
40 [0019] With the coordinate transformation performed with Formula (1), all of the image size enlargement or reduc- 
tion of the person image represented by the original image signal SO, the rotation of the entire person image, and the 
parallel translation of the person image are achieved simultaneously. 

[0020] The coefficients a, b, c, and d in Formula (1) are determined in the manner described below. Specifically, 
Formula (1) can be divided into Formulas (2) and (3) shown below. 

45 

x2=ax1+by1+c (2) 

y2=-bx1+ay1+d (3) 

so The two points in the person image, which are detected by the pattern matching process as the points having the high- 
est degree of correlation with the reference image, may have coordinates (x1 1, y1 1) and coordinates (x12, y12). Also, 
the two points in the reference image, which correspond to the two points in the person image, may have coordinates 
(x21, y21) and coordinates (x22, y22). In such cases, Formulas (4), (5), (6), and (7) shown below obtain. 

55 x21=ax11+by11+c (4) 

x22=ax12+by12+c (5) 
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y21=-bx1 1+ay1 1+d (6) 
y22=-bx1 2+ay 1 2+d (7) 

s The four coefficients a, b, c, and d can be calculated by solving Formulas (4), (5), (6), and (7). 

[0021] How this embodiment operates will be described hereinbelow. Figure 3 is a flow chart showing how the 
embodiment of the image processing apparatus in accordance with the present invention operates. 
[0022] With reference to Figure 3, firstly, in a step S1 , the reading means 1 reads the original image signals SO, SO, 
... from the storage medium or the storage device. In a step S2, each of the original image signals SO, SO, ... is fed into 

10 the detection means 2. In a step S3, the edge detecting means 6 of the detection means 2 performs a differentiation 
process, or the like, on the original image signal SO and detects edges in the person image represented by the original 
image signal SO. In a step S4, the binarization means 7 performs a binarizing process on the original image signal SO, 
which has been obtained from the edge detection, the binary signal SO' is obtained from the binarizing process. There- 
after, in a step S5, the pattern matching means 8 calculates the displacement quantities, which are to be utilized for 

15 transforming the center position having the coordinates (x, y) in the face pattern embedded in the person image into the 
center position having the coordinates (xO, yO) in the reference image, and uniformizing the size v of the face pattern in 
the vertical direction and the size h of the face pattern in the horizontal direction. The information, which represents the 
calculated displacement quantities, is fed as the face information H into the normalization means 3. 
[0023] In a step S6, the normalization means 3 calculates the affine transformation coefficients with Formula (1) 

20 and in accordance with the face information H, performs the affine transformation on the original image signal SO, and 
obtains the normalized image signal S1 representing the image, in which the position and/or the size of the face pattern 
of the person has been normalized. In a step S7, a judgment is made as to whether ail of the original image signals SO, 

50, ... having been read by the reading means 1 have been or have not been normalized. In cases where it has been 
judged that all of the original image signals SO, SO, ... have not been normalized, the processes from the step S2 to the 

25 step S6 are iterated. In cases where it has been judged that all of the original image signals SO, SO, ... have been nor- 
malized, in a step S8, the editing means 4 lays out the images, which are represented by the normalized image signals 

51 , S1 , in a predetermined layout and obtains the layout image signal S2 representing the layout image R. Thereaf- 
ter, in a step S9, the printer 5 prints the layout image R in accordance with the layout image signal S2. 

[0024] In this manner, with this embodiment, the process for normalizing the face patterns of the persons in the per- 
30 son images is performed on the original image signals SO, SO, ... representing the person images, and the normalized 
image signals S1 , SI , ... are thereby obtained. The images, which are represented by the normalized image signals S1 , 

51, are laid out in the predetermined layout, and the layout image signal S2 representing the thus formed layout 
image is obtained. Therefore, as illustrated in Figure 4, in the layout image R reproduced from the layout image signal 

52, the positions and/or the sizes of the face patterns of the persons become uniform. Accordingly, the layout image R, 
35 in which the positions and/or the sizes of the face patterns of the persons are uniform, can be formed efficiently. As a 

result, the productivity of layout images R, R, .... such as school photos, can be enhanced. 

[0025] In the embodiment described above, the detection means 2 detects at least two points having the highest 
degree of correlation by performing the pattern matching process and calculates the affine transformation coefficients 
a, b, c, and d to be utilized for the normalization of the position and/or the size of the face pattern of the person in the 
40 person image. In cases where only the position of the face pattern of the person in the person image is to be compen- 
sated for, only one point having the highest correlation may be detected, and the position of the face pattern may be 
uniformized in accordance with the detected point. In this manner, the normalization of the face pattern can be per- 
formed quickly. 

[0026] Also, in the embodiment described above, the detection means 2 detects the edges in the person image, 
45 performs the pattern matching with the reference image, and thereby calculates the displacement quantities. Alterna- 
tively, the displacement quantities may be calculated with a technique described in, for example, Japanese Unexamined 
Patent Publication No. 6(1 994)-67320. Specifically, the person image may be divided into a plurality of regions in 
accordance with the distributions of the hue and saturation values in the person image, and a face candidate region may 
be detected. A face region may then be detected in accordance with the shape of a neighboring region, which is located 
so in the vicinity of the face candidate region. Thereafter, the pattern matching process as described above may be per- 
formed on the signal representing the face region, and the displacement quantities may thereby be calculated. As 
another alternative, an ellipse circumscribed about a simply detected face candidate region may be formed, and the dis- 
placement quantities with respect to the reference image may be calculated by taking the center point of the ellipse as 
the center position of the face pattern having the coordinates (x, y). The face region may be detected with one of various 
55 other techniques. For example, the face region may be detected by utilizing neural network techniques described in, for 
example, Japanese Unexamined Patent Publication Nos. 5(1993)-274438 and 5(1993)-307605. 
[0027] Further, in the aforesaid embodiment, the layout image R is formed by normalizing the face patterns of a plu- 
rality of different persons. Alternatively, as illustrated in Figure 6, an album may be formed by combining a plurality of 
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layout images, each of which comprises a plurality of images of the same person. In such cases, the image processing 
apparatus in accordance with the present invention can be utilized for normalizing the positions and/or the sizes of the 
face patterns, which are embedded in the images constituting each layout image. 

s Claims 

1 . An image processing method for obtaining a layout image signal representing a layout image, in which a plurality 
of person images are laid out, from a plurality of original image signals, each of the original image signals repre- 
senting a person image, in which a face pattern of a person is embedded, the method comprising the steps of: 

i) detecting face information from each of the original image signals, said face information representing a posi- 
tion and/or a size of the face pattern of the person in the person image represented by each original image sig- 
nal,' 

ii) performing a face pattern normalizing process on each of the original image signals and in accordance with 
is said detected face information, a plurality of normalized image signals being obtained from said face pattern 

normalizing process, and 

Hi) laying out a plurality of images, which are represented by said normalized image signals, in a predetermined 
layout, whereby the layout image signal representing the thus formed layout image is obtained. 

20 2. A method as defined in Claim 1 wherein said face pattern normalizing process is performed by utilizing affine trans- 
formation. 

3- An image processing apparatus for obtaining a layout image signal representing a layout image, in which a plurality 
of person images are laid out, from a plurality of original image signals, each of the original image signals repre- 
ss senting a person image, in which a face pattern of a person is embedded, the apparatus comprising: 

i) detection means for detecting face information from each of the original image signals, said face information 
representing a position and/or a size of the face pattern of the person in the person image represented by each 
original image signal, 

30 ii) normalization means for performing a face pattern normalizing process on each of the original image signals 

and in accordance with said detected face information, a plurality of normalized image signals being obtained 
from said face pattern normalizing process, and 

iii) editing means for laying out a plurality of images, which are represented by said normalized image signals, 
in a predetermined layout, and obtaining the layout image signal representing the thus formed layout image. 

35 

4. An apparatus as defined in Claim 3 wherein said face pattern normalizing process is performed by utilizing affine 
transformation. 

5. A recording medium, on which a program for causing a computer to execute an image processing method has been 
40 recorded and from which the computer is capable of reading the program, the image processing method compris- 
ing obtaining a layout image signal representing a layout image, in which a plurality of person images are laid out, 
from a plurality of original image signals, each of the original image signals representing a person image, in which 
a face pattern of a person is embedded, 

wherein the program comprises the procedures of: 

i) detecting face information from each of the original image signals, said face information representing a posi- 
tion and/or a size of the face pattern of the person in the person image represented by each original image sig- 
nal, . /•. ■■• ••••• . •=.-.'i.- : '=i- s *. •.■««■.' ' ■\-':'\-'^y'\-- :or -: ■,v : r-.' 

ii) performing a face pattern normalizing process on each of the original image signals and in accordance with 
so said detected face information, a plurality of normalized image signals being obtained from said face pattern 

normalizing process, and 

iii) laying out a plurality of images, which are represented by said normalized image signals, in a predetermined 
layout, whereby the layout image signal representing the thus formed layout image is obtained. 

55 6. A recording medium as defined in Claim 5 wherein said face pattern normalizing process is performed by utilizing 
affine transformation. 
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